Research Cell: An International Journal of Engineering Sciences

Urvashi Garg ¹, Vishal Goyal ²

Affiliations
1 Haryana College of Technology and Management, Kaithal, IN
2 Punjabi University, Patiala, IN

Abstract

Stop word removal is one of the important NLP techniques. Stop words are very common in any document. In this paper, we have created a list of stop words for Hindi text on the basis of frequency of words in documents. Hindi documents from EMILLE corpus have been used for finding out the stop words. UTF-8 encoding is used. The percentage of stop words in any document has been find out and experimentally analyzed. The paper discusses the effect of stop word removal on the similarity of two documents containing Hindi text. Hoad&Zobel approach is used for finding the similarity of documents containing Hindi text.

Keywords

Stop Words, Removal, Text, Hindi, List, Frequency.

Full Text

   

Rupinder Pal Kaur ¹, Vishal Goyal ²

Affiliations
1 Post Graduate Department of Computer Science, Sri Guru Gobind Singh College, Sector 26, Chandigarh, IN
2 Department of Computer Science, Punjabi University, Patiala, IN

Abstract

Quality is difficult to define and measure. ISO 25010 standard describes the model 'quality in use' for evaluating the usability quality of software. In this research a tool based on ISO 25010 standard has been developed specifically to test 'Quality in Use' for websites of Punjabi and Hindi newspapers. The attributes that affect the usability quality are recognized and arranged in a questionnaire format in preparation for data collection. The data analysis can be conducted by using some statistical tool on the weighted dataset, where the rating for a question for each respondent can be multiplied by its perceived importance. The tool developed can be used to obtain feedback that can help understand, control, improve and make predictions about websites of newspapers in other languages also.

Keywords

Quality in Use, Newspaper Domain, Questionnaire, ISO 25010, Usability Quality, Quality Attributes, Websites of Punjabi and Hindi Newspapers.

Full Text

   

Shruti Bajaj Mangal ¹, Vishal Goyal ²

Affiliations
1 Punajbi University, Patiala, IN
2 Punjabi University, Patiala, IN

Abstract

This paper describes the Naive Bayes text News classification system developed for Punjabi Language. News corpus is used for training and testing purpose of the classifiers. Language specific preprocessing techniques are applied on raw data to generate a standardized and reduced-feature lexicon. Punjabi language is morphological rich language which makes those tasks complex. Statistical characteristics of corpus and lexicon are measured which show satisfactory results of text preprocessing module. We are able to get satisfactory results using Naive Bayes Classifier.

Keywords

Naive Bayes, Text classification, Punjabi.

Full Text

   

Umrinder Pal Singh ¹, Vishal Goyal ¹, Anisha Rani ¹

Affiliations
1 Department of Computer Science, Punjabi University, Patiala, Punjab, IN

Abstract

Word sense disambiguation is widely studied and discussed area of NLP for any natural language under consideration. Words have different senses. The task of selecting the correct sense for a word is called word sense disambiguation. In Machine Translation the problem when we translate the ambiguous words. To resolve this problem we develop Word Sense Disambiguation module that resolve the problem of ambiguity of Hindi words of a particular sentence. Deleted Interpolation and Back-Off, N-gram smoothing models are used to implement Word Sense Disambiguation. we have used the tri-gram to implement these models. we use a dictionary based approach.

Keywords

Word Sense Disambiguation, Punjabi, Back-Off, Deleted Interpolation.

Full Text

   

Manish Kumar ¹, Vishal Goyal ², Jaskaran Bhullar ³

Affiliations
1 Punjab Technical University, Jalandhar, IN
2 Department of Computer Science, Punjabi University, Patiala, IN
3 MIMIT, Malout

Abstract

Technical education plays vital role in the development of any state or country. Language also plays the major role in the educational development of any region. India is a Multi-lingual country where every region has their own prominent local language, which is the primary mode of communication and is also mode of basic education. On the other hand English plays the dominating role for providing Technical education in Punjab. In this paper we study the scope and challenge to develop a domain based Machine Translation system which can translate technical terms from Hindi to Punjabi to break the language barrier.

Keywords

Machine Translation, Optical Character Recognition, Higher Education, Hindi, Punjabi Technical Ambiguity, Non-Technical Ambiguity.

Full Text

   

Rakesh Kumar ¹, Vishal Goyal ², Ravinder Khanna ³

Affiliations
1 Punjab Technical University, Kapurthala, IN
2 Dept. of Computer Science, Punjabi University, Patiala, IN
3 MM University, Sadopur, Ambala, IN

Abstract

India has many regional languages. Attempts have been made for developing machine translations between these languages, but little success has been reported so far. Analysis of Hindi to Punjabi machine translation system devised by Punjabi University, Patiala, India has found that Hindi post position से (sē) is translated inaccurately being its ambiguous nature, most of the times, as it has eighteen different senses in Punjabi. The overall translation success rate of this system reported as 87.60%, however the translation success rate in respect of this post position से (sē) is only about 2%. In this paper, N-gram approach (along with its smoothing variants) has been applied to improve the accuracy of translation of this post position से (sē) in already developed Hindi to Punjabi Machine Translation System. It has been concluded that bigram approach with Add-One smoothing algorithm gives the best results in improving the accuracy of translation of post position से (sē) from 2% to 85.49%, thus improving the overall machine translation accuracy of the system from 87.60% to 92.30% .

Keywords

Natural Language Processing (NLP), Word Sense Disambiguation (WSD), Machine Translation (MT).

Full Text

   

Harjeet Singh ¹, Vishal Goyal ², Ravinder Khanna ³

Affiliations
1 Punjab Technical University, Jalandhar, IN
2 Dept. of Computer Science, Punjabi University, Patiala, IN
3 MM University, Sadopur, Ambala, IN

Abstract

A lot of work is done on the standard varieties of Indian languages in the field of MT but dialectal variety of a language is still an unexplored area. Machine Translation between a Standard language and its dialect is easy as standard variety and its dialect are closely related to each other. When the Language pair is closely related then due to the common grammar and vocabulary, it become easy to develop a MT system.

In Punjab generally Malwai, Majhi, Doabi and Powadi dialect are used for the oral communication and no work is still done on any dialect in the field of Machine Translation. This Paper discusses the various phases in the development of Machine Translation system for Standard Punjabi - Malwai dialect pair.

Keywords

Machine Translation, Malwai Dialect.

Full Text

   

Rakesh Kumar ¹, Ravinder Khanna ², Vishal Goyal ³

Affiliations
1 Punjab Technical University, Kapurthala, IN
2 Sachdeva Engg. College for Girls, Gharuan, Mohali, IN
3 Dept. of Computer Science, Punjabi University, Patiala, IN

Abstract

Artificial intelligence (AI) has been a major research area in the later quarter of 20^th century and is likely to be even more so in the 21^st century. A key part of AI is Word Sense Disambiguation (WSD) which deals with choosing the correct sense of a word in the given text. All human languages have words with multiple meaning and selecting the intended sense is important. This paper briefly describes various methods presently used for WSD and their relative effectiveness. WSD applications currently find application in Information Retrieval, Information Extraction, Automated Answering Machine, Speech Reorganization, Machine Translation among many others. WSD has promise for the future in taking AI to the next higher level.

Keywords

Natural Language Processing (NLP), Artificial Intelligence (AI), Word Sense Disambiguation (WSD), Knowledge Based Methods, Supervised/Unsupervised Methods.

Full Text

   

Rajeev Puri ¹, R. P. S. Bedi ¹, Vishal Goyal ²

Affiliations
1 Punjab Technical University, Kapurthala Road, Jalandhar, IN
2 Dept of Comp. Sc, Punjabi University, Patiala, IN

Abstract

Plagiarism detection has always been a challenging task for researchers and developers across the world. A number of researchers have contributed to this task by suggesting methods to detect plagiarism in research publications, books and software. Subsequently some applications based on these methods are also available in open source as well as commercial marketplace. Majority of these applications are performing well with English language documents, but fail to give satisfactory results with documents written in regional languages. This paper deals with investigating the challenges in plagiarism detection in regional languages, with specific case study of documents written in Punjabi language.

Full Text

   

Manish Kumar ¹, Vishal Goyal ², Jaskaran Bhullar ³

Affiliations
1 Punjab Technical University, Jalandhar, Punjab, IN
2 Department of Computer Science, Punjabi University, Patiala, Punjab, IN
3 MIMIT, Malout, Punjab, IN

Abstract

Present paper is the report of the survey conducted to make a consensus whether technical education at Higher level should be provided in mother tongue in the state of Punjab. Survey has been conducted with primary data collected through structured questionnaire. Questionnaire was filled by various stake holders like students and teachers. Data was analyzed by the help of percentile, mean and presented by tables and figures.

Keywords

Technical Education, Translation, Transliteration, Survey, Mother Tongue, Medium of Instruction.

Full Text

   

Harjeet Singh ¹, Ravinder Khanna ², Vishal Goyal ³

Affiliations
1 Punjab Technical University, Jalandhar, IN
2 Department of Electronics, M.M. University, Ambala, IN
3 Department of Computer Science, Punjabi University, Patiala, IN

Abstract

Punjabi language is an modern Indo-Aryan language. It is 10th most spoken language in the world. Most of the speakers of Punjabi language lives in Punjab region of India. Punjabi has mainly four dialects viz Majhi, Doabi, Malwi and Pwadhi. Today, dialects represents the default variety of oral communication in the Punjab and Standard Punjabi is almost exclusively used for writing. The Malwai dialect is taken for the study because there are twenty two districts in Punjab state and the malwai dialect is spoken in more than half of the districts. Both Standard Punjabi and Malwai use the same script i.e. Gurukhi but the dialect is different. In this paper the difference between the Standard Punjabi and Malwai are explained.

Keywords

Machine Translation, Punjabi, Malwai Dialect.

Full Text

   

Rajeev Puri ¹, R. P. S. Bedi ¹, Vishal Goyal ²

Affiliations
1 Punjab Technical University, Kapurthala Road, Jalandhar, IN
2 Dept of Comp. Sc, Punjabi University, Patiala, IN

Abstract

Many information retrieval tasks deal with the classification of huge amount of data before giving final results. The data being processed in IR tasks may or may not be useful for the researchers. There has to be some method to identify such data (called stop words) and remove it from data set before beginning with the IR task. This gives dual benefits – Reducing the overall vector space, thereby leading to performance improvements in terms of execution speed and the relevance of results. The purpose of this paper is to find a suitable, automated method for identification of stop words in Punjabi Text.

Keywords

Punjabi Stop Words List, Statistical Modeling, Borda Count, Information Processing, Text Classification.

Full Text

   

Ajit Kumar ¹, Vishal Goyal ²

Affiliations
1 Multani Mal Modi College, Patiala, IN
2 Department of Computer Science, Punjabi University, Patiala, IN

Abstract

India is a country with huge population of over hundred and twenty crore, who speak different languages. Only 5% of Indian population can effectively communicate in English and rest 95% are comfortable with their regional languages and are deprived of the benefits of information technology. To penetrate the benefits of Communication and Information Technology up to common masses, Ministry of Communication and Information Technology, Government of India, initiated a Programme known as Technology Development for Indian Languages (TDIL). The objective of the programme is to develop Information Processing Tools and Techniques to facilitate human-machine interaction without language barrier; to create and access multilingual knowledge resources; and integrate them to develop innovative user products and services. This paper surveys the various initiatives taken under TDIL for the promotion of Communication and Information Technology in Indian Languages. We are also providing an introductory list of tools/products developed by different institutions for Indian languages.

Full Text

   

Deepali ¹, Vishal Goyal ¹, Lalit Goyal ²

Affiliations
1 Department of Computer Science, Punjabi University, Patiala (Punjab), IN
2 Department of Computer Science, DAV College, Jalandhar (Punjab), IN

Abstract

.This paper outlines the concept for translation of English text to Indian Sign Language using real-domain synthetic animations. The translation framework consists of a processing module that parses the English input sentence to phrase structure grammar representation on which Indian sign language grammar rules are applied to reorder English phrase terms. The input English sentences was given by the user are parsed through the converter module which in turn exchange the complex and compound English sentences to their simplify versions by means of complex to simple and compound to simple English grammar rules. Elimination module removes unacceptable terms from the reordered sentence. Lemmatization is done in order to translate the words into the ischolar_main form since the Indian sign language does not use the word inflections. All the words in the sentence are then tested into a lexicon containing the English word and its HamNoSys notation and their synonym replaces the terms that are not in the lexicon. Sentence words are substituted with their HamNoSys counter code. In the event that the word is not in the lexicon, HamNoSys code is taken for each word alphabet. The HamNoSys data is translated to the SiGML tags, and these SiGML tags are sent to animation module which translates the SiGML code to the synthetic animation utilizing avatar.

Keywords

Indian Sign language ,HamNoSys,SiGML

Full Text

   

Kapil Dev Goyal ¹, Vishal Goyal ¹

Affiliations
1 Department of Computer Science, Punjabi University, Patiala, IN

Abstract

Multiword Expressions (MWEs) play a vital role in Natural Language Processing. Multiword Expression is a combination of two or more words but treated as a single word. In Punjabi Language, there are varieties of MWEs and many of these are of the types that are not found in English. In this paper, we discuss different types of MWEs encountered in Punjabi. For example, replicated words, word combination with antonym, synonym, hyponym, gender, number and ‘waala’ morpheme have not been discovered as MWEs in English. Rule based approachs, statistical methods, and linguists’ approaches were used for MWE identification and extraction. In this paper, we present a methodology for identification and extraction of Punjabi MWEs using statistical methods, rule base methods and linguists’ approach.

Full Text

   

Mukhtiar Singh ¹, Vishal Goyal ¹, Sahil Raj ²

Affiliations
1 Department o f Computer Science, Punjabi University, Patiala, IN
2 School o f Management Studies, Punjabi University, Patiala, IN

Abstract

Sentiment analysis is a field of study for analyzing emotions of people such as happy, sad, angry, etc. towards the entities and attributes expressed in written text. In this study, the data was collected in the textual form from different sources like Facebook, YouTube, Twitter, and Whatsapp, then pre-processed the collected data. After that, identification of the language of code-mixed text performed, which includes tokenization, word-play, misspelled words, abbreviations, slang words, phonetic-typing, etc. After the identification task, the English-Punjabi dictionary was created which was consisting of opinionated words list like positive, negative, and neutral words list. The rest of the words are being stored in an unsorted word list. In the last, a statistical technique applied at sentence level sentiment polarity of the English-Punjabi code mixed dataset. It was identified that the results up to the Five-Grams and Tri-Grams approaches had the similarity.

Keywords

Code Mixed Text, Romanized Text, Natural Language Processing, Text Processing, Romanized Text, Sentiment Analysis, Microblogging.

Full Text

   

Neetika Bansal ¹, Vishal Goyal ², Simpel Rani ³

Affiliations
1 Department of Computer Science, Punjabi University College of Engineering & Management, Rampura Phul, IN
2 Department of Computer Science Punjabi University, Patiala, IN
3 Department of Computer Science and Engineering, Yadavindra College of Engineering, Talwandi Sabo, IN

Abstract

Code mixing denotes using multiple languages in an utterance. It is clearly seen that code mixing is pervasive while people communicate over social media irrelevant of the mode being used. The fusion of languages makes it more challenging and requires consistent updates according to recent trends. The current paper addresses three approaches namely CRFs (Conditional Random Fields), Bi-LSTM (Long Short-term Memory) and CNNs( Convolutional Neural Networks). Firstly, for word-level language identification of code-mixed English-Punjabi text CRF based system uses lexical, contextual, character ngram, and special character features. Secondly, Recursive Neural Network namely Bi-LSTM with glove embedding is used for language identification and thirdly CNN with glove embedding is used for language identification. It is observed that CRFs is the best performing system with an f1-score of 0.96.

Keywords

Code Mixing, Language Identification, Deep Learning, Glove Embedding, Conditional Random Fields.

Full Text

   

Maajid Bashir ¹, Vishal Goyal ², Kaiser J.Giri ³

Affiliations
1 Department of Computer Science, Punjabi University, Patiala,147002, Punjab, India., IN
2 Department of Computer Science, Punjabi University, Patiala, 147002, Punjab, India., IN
3 Department of Computer Science, Islamic University of Science and Technology, Awantipora, 192122,Jammu and Kashmir, IN

Abstract

Optical Character Recognition, often known as OCR, is a method that turns scanned documents, images of text, and PDFs into text documents, that can be edited and searched on a computer. OCR software analyzes a scanned image of text and turns it into machine-encoded text by identifying the characters in the image and transforming them into a digital format. Acknowledging the relevance of optical character recognition (OCR) in the actual world, a multitude of approaches have evolved both for Western and Asian languages. Kashmiri is mostly spoken in the Kashmir Valley, which is located in Jammu and Kashmir India. In spite of the significant amount of effort that has been done into recognizing Indian scripts such as Devanagari, Bengali, Urdu, and Punjabi, no such effort has been made to recognize Kashmiri script. In addition, several benchmark corpora for other Perso Arabic scripts, such as Urdu, Arabic, and Pashto, have been developed for the purpose of training and assessing various OCR systems. Notably, there is currently no OCR corpus for Kashmiri script that can be utilized to train and evaluate deep neural networks for the development of Kashmir OCR. To that purpose, we have proposed a Kashmiri corpus Printed Kashmiri Image Text (PKIT)consisting of 120000 line, and 523000-word level printed text images respectively, well suited for use in deep learning techniques. Additionally, we used the proposed dataset for training different state of art deep learning approaches thereby obtaining the Word Error Rate (WER) and Character Error Rate ((CER)of 5.62% on average.

Keywords

Dataset Generation, Deep Learning, Kashmiri OCR, Optical Character Recognition, Printed Kashmiri Text Recognition.

Full Text